Single image super resolution for spatial enhancement of hyperspectral remote sensing imagery

Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations.Hyperspectral Imaging (HSI) has emerged as a powerful tool for capturing detailed spectral information across various applications, such as remote sensing, medical imaging, and material identification. However, the limited spatial resolution of acquired HSI data poses a challenge due to hardware and acquisition constraints. Enhancing the spatial resolution of HSI is crucial for improving image processing tasks, such as object detection and classification. This research focuses on utilizing Single Image Super Resolution (SISR) techniques to enhance HSI, addressing four key challenges: the efficiency of 3D Deep Convolutional Neural Networks (3D-DCNNs) in HSI enhancement, minimizing spectral distortions, tackling data scarcity, and improving state-of-the-art performance. The thesis establishes a solid theoretical foundation and conducts an in-depth literature review to identify trends, gaps, and future directions in the field of HSI enhancement. Four chapters present novel research targeting each of the aforementioned challenges. All experiments are performed using publicly available datasets, and the results are evaluated both qualitatively and quantitatively using various commonly used metrics. The findings of this research contribute to the development of a novel 3D-CNN architecture known as 3D Super Resolution CNN 333 (3D-SRCNN333). This architecture demonstrates the capability to enhance HSI with minimal spectral distortions while maintaining acceptable computational cost and training time. Furthermore, a Bayesian-optimized hybrid spectral spatial loss function is devised to improve the spatial quality and minimize spectral distortions, combining the best characteristics of both domains. Addressing the challenge of data scarcity, this thesis conducts a thorough study on Data Augmentation techniques and their impact on the spectral signature of HSI. A new Data Augmentation technique called CutMixBlur is proposed, and various combinations of Data Augmentation techniques are evaluated to address the data scarcity challenge, leading to notable enhancements in performance. Lastly, the 3D-SRCNN333 architecture is extended to the frequency domain and wavelet domain to explore their advantages over the spatial domain. The experiments reveal promising results with the 3D Complex Residual SRCNN (3D-CRSRCNN), surpassing the performance of 3D-SRCNN333. The findings presented in this thesis have been published in reputable conferences and journals, indicating their contribution to the field of HSI enhancement. Overall, this thesis provides valuable insights into the field of HSI-SISR, offering a thorough understanding of the advancements, challenges, and potential applications. The developed algorithms and methodologies contribute to the broader goal of improving the spatial resolution and spectral fidelity of HSI, paving the way for further advancements in scientific research and practical implementations

Autonomous building detection using region properties and PCA

This paper proposes an algorithm for autonomous building detection in remote sensing images. The basis of the algorithm relies on the fact that each channel in RGB color space conveys different information. Furthermore, region properties and Principal Component Analysis (PCA) are used to distinguish between buildings and other regions in order to reduce false positive cases. The images used to test the proposed algorithm were obtained from DubaiSat-2, which offers multispectral images with 1-m accuracy

SDF2Net: Shallow to Deep Feature Fusion Network for PolSAR Image Classification

Polarimetric synthetic aperture radar (PolSAR) images encompass valuable information that can facilitate extensive land cover interpretation and generate diverse output products. Extracting meaningful features from PolSAR data poses challenges distinct from those encountered in optical imagery. Deep learning (DL) methods offer effective solutions for overcoming these challenges in PolSAR feature extraction. Convolutional neural networks (CNNs) play a crucial role in capturing PolSAR image characteristics by leveraging kernel capabilities to consider local information and the complex-valued nature of PolSAR data. In this study, a novel three-branch fusion of complex-valued CNN, named the Shallow to Deep Feature Fusion Network (SDF2Net), is proposed for PolSAR image classification. To validate the performance of the proposed method, classification results are compared against multiple state-of-the-art approaches using the airborne synthetic aperture radar (AIRSAR) datasets of Flevoland and San Francisco, as well as the ESAR Oberpfaffenhofen dataset. The results indicate that the proposed approach demonstrates improvements in overallaccuracy, with a 1.3% and 0.8% enhancement for the AIRSAR datasets and a 0.5% improvement for the ESAR dataset. Analyses conducted on the Flevoland data underscore the effectiveness of the SDF2Net model, revealing a promising overall accuracy of 96.01% even with only a 1% sampling ratio

Operational Neural Networks for Efficient Hyperspectral Single-Image Super-Resolution

Hyperspectral Imaging is a crucial tool in remote sensing which captures far more spectral information than standard color images. However, the increase in spectral information comes at the cost of spatial resolution. Super-resolution is a popular technique where the goal is to generate a high-resolution version of a given low-resolution input. The majority of modern super-resolution approaches use convolutional neural networks. However, convolution itself is a linear operation and the networks rely on the non-linear activation functions after each layer to provide the necessary non-linearity to learn the complex underlying function. This means that convolutional neural networks tend to be very deep to achieve the desired results. Recently, self-organized operational neural networks have been proposed that aim to overcome this limitation by replacing the convolutional filters with learnable non-linear functions through the use of MacLaurin series expansions. This work focuses on extending the convolutional filters of a popular super-resolution model to more powerful operational filters to enhance the model performance on hyperspectral images. We also investigate the effects that residual connections and different normalization types have on this type of enhanced network. Despite having fewer parameters than their convolutional network equivalents, our results show that operational neural networks achieve superior super-resolution performance on small hyperspectral image datasets.Comment: 12 pages, 7 figure

Super resolution of DS-2 satellite imagery using deep convolutional neural network

Nowadays, Satellite images are used for various analysis, including building detection and road extraction, which are directly beneficial to governmental applications, such as urbanization and monitoring the environment. Spatial resolution is an element of crucial impact on the usage of remote sensing imagery. High spatial resolution means satellite images provide more detailed information. To improve the spatial resolution at the sensor level, many factors are ought to be taken into consideration, such as the manufacturing process. Moreover, once the satellite is launched, no further action can be taken from the perspective of hardware. Therefore, a more practical solution to improve the resolution of a satellite image is to use Single Image Super Resolution (SISR) techniques. This research proposal deals with the re-design, implementation, and evaluation of SISR technique using Deep Convolutional Neural Network with Skip Connections and Network in Network (DCSCN) for enlarging multispectral remote sensing images captured by DubaiSat-2 (DS-2) and estimating the missing high frequency details. The goal is to achieve high performance in terms of quality, and to test whether training the network using luminance channel only, which is extracted from YCbCr domain, can achieve high quality results. For this purpose, DCSCN is trained, evaluated, and tested using a dataset collected from DS-2. A single low resolution DS-2 image is used to construct its high resolution version by training the model from scratch and fine-tuning its hyper-parameters to produce optimal results. The performance is evaluated using various quality indices, such as Structural Similarity Index Measurement (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Wavelet domain Signal-to-Noise Ratio (WSNR). The performance is compared to other state-of-the-art methods, such as Bil-inear, Bi-cubic, and Lanczos interpolation

Autonomous palm tree detection from remote sensing images-UAE dataset

Autonomous detection and counting of palm trees is a research field of interest to various countries around the world, including the UAE. Automating this task saves effort and resources by minimizing human intervention and reducing potential errors in counting. This paper introduces a new High Resolution (HR) remote sensing dataset for autonomous detection of palm trees in the UAE. The dataset is collected using Unmanned Aerial Vehicles (UAV), and it is labeled properly in PASCAL VOC and YOLO formats after preprocessing and visually inspecting its quality. A comparative evaluation between Faster-RCNN and YOLOv4 networks is then conducted to observe the usability of the dataset in addition to the strengths and weaknesses of each network. The dataset is publicly available at https://github.com/Nour093/Palm-Tree-Dataset

Spatio-temporal analysis and machine learning for traffic accidents prediction

Traffic accidents impose significant problems in our daily life due to the huge social, environmental, and economic expenses associated with them. The rapid development in data science, geographic data collection, and processing methods encourage researchers to evaluate, delineate traffic accident hotspots, and to effectively predict and estimate traffic accidents. In this study, traffic accidents dataset that covers United Kingdom for the time period between 2012-2014 is investigated. The methodology consists of extracting features weights, and then using these weights with statistical methods provided in ArcGIS in order to classify accidents according to severity and perform hotspot analysis and severity prediction. The proposed method can be effectively used by different authorities to implement an improved planning and management approaches for traffic accident reduction. Moreover, it can identify and locate road risk segments where immediate action should be considered

A review of spatial enhancement of hyperspectral remote sensing imaging techniques

Remote sensing technology has undeniable importance in various industrial applications, such as mineral exploration, plant detection, defect detection in aerospace and shipbuilding, and optical gas imaging, to name a few. Remote sensing technology has been continuously evolving, offering a range of image modalities that can facilitate the aforementioned applications. One such modality is Hyperspectral Imaging (HSI). Unlike Multispectral Images (MSI) and natural images, HSI consist of hundreds of bands. Despite their high spectral resolution, HSI suffer from low spatial resolution in comparison to their MSI counterpart, which hinders the utilization of their full potential. Therefore, spatial enhancement, or Super Resolution (SR), of HSI is a classical problem that has been gaining rapid attention over the past two decades. The literature is rich with various SR algorithms that enhance the spatial resolution of HSI while preserving their spectral fidelity. This paper reviews and discusses the most important algorithms relevant to this area of research between 2002-2022, along with the most frequently used datasets, HSI sensors, and quality metrics. Meta-analysis are drawn based on the aforementioned information, which is used as a foundation that summarizes the state of the field in a way that bridges the past and the present, identifies the current gap in it, and recommends possible future directions

SISR of hyperspectral remote sensing imagery using 3D encoder-decoder RUNet architecture

Single Image Super Resolution (SISR) refers to the spatial enhancement of an image from a single Low Resolution (LR) observation. This topic is of particular interest to remote sensing community, especially in the area of Hyperspectral Imagery (HSI) due to their high spectral resolution but limited spatial resolution. Enhancing the spatial resolution of HSI is a pre-requisite that boosts the accuracy of other image processing tasks, such as object detection and classification. This paper deals with SISR of HSI through the 3D expansion of Robust UNet (RUNet). The network is developed, trained, and tested over two datasets, and compared against the original 2D-RUNet and other state-of-the-art approaches. Quantitative and qualitative evaluation show the superiority of 3D-RUNet and its ability to preserve the spectral fidelity of the enhanced HSI

Automatic detection of potential buried archaeological sites in Saruq Al-Hadid, United Arab Emirates

The use of remote sensing in archaeological research allows the prospection of sub-surfaces in arid regions non- intrusively before the on-site investigation and excavation. While the actual detection method of expected buried archaeological structures is based on visual interpretation, this work provides a supporting archaeological guidance using remote sensing. The aim is to detect potential archaeological remains underneath the sand. This paper focuses on Saruq Al-Hadid surroundings, which is an archaeologist site discovered in 2002, located about 50 km south-east of Dubai, as archaeologists believe that other archaeological sites are potentially buried in the surroundings. The input data is derived from a combination of wavelength L-band Synthetic Aperture Radar (ALOS PALSAR), which is able to penetrate the sand, and multispectral optical images (Landsat 7). This paper develops a new strategy to help in the detection of suspected buried structures. The data fusion of surface roughness and spectral indices enables tackling the well-known limitation of SAR images and offers a set of pixels having an archaeological signature different from the manmade structures. The potential buried sites are then classified by performing a pixel-level unsupervised classification algorithm such as K-means cluster analysis. To test the performance of the proposed method, the results are compared with those obtained by visual interpretation